This invention relates to digital image processing and in particular to automatically processing a digital image to produce a visual color reproduction of a scene. More specifically, the invention relates to a visual color reproduction of a scene having preferred color reproduction and scene-dependent tone scaling.
Color image reproduction methods and systems known in the art capture images on image-receptive media, which can be stored in analog or digital form, and then output as a visual reproduction. For example, color images may be captured on photographic negative film and then reproduced optically on photographic paper. Images can also be captured on positive photographic media, and then viewed directly, or copied onto other transparent and reflective media. In addition, color negative films, transparency films or reflective prints can be scanned for input to digital imaging systems. Subsequently, digital color and tone manipulations can be applied to the digital picture element (pixel) values in order to produce the best possible reproduction for the intended output device and medium, and the resulting images can be viewed on monitors, printed on silver halide photographic paper or on other reflective media using inkjet, dye sublimation or electrophotographic printers. Digital images can also be encoded in a defined color space and stored on various media, e.g. Kodak Photo CD, Kodak Picture Disk or CD, at any point in this sequence for future processing. In other cases, color images can be captured by electronic devices, such as video or still CCD cameras, and viewed on monitors or printed using inkjet or dye sublimation thermal printers.
In each case previously cited, these systems are subjected to customer satisfaction criteria and may or may not embody digital tone sale reproduction manipulation or some form of color enhancement. The systems mentioned above are just some examples of color image reproduction systems.
It is well known in the art, that the best reproductions of original scenes do not constitute a 1:1 mapping of scene colorimetry. For example, the correct scaling of lightness and chroma values depends on the viewing conditions of the original scene and the reproduction. For the purpose of this discussion, viewing conditions are defined as the overall luminance level of the scene or reproduction, the relative brightness of the surround, the state of chromatic adaptation of the observer and the amount of stray light (flare) present. Equivalent color has been defined, as a reproduction, in which the chromaticities, relative luminances and absolute luminances are such that, when seen in the picture-viewing conditions, they have the same appearance as the original scene. This type of match is addressed by color appearance models. It has been argued that equivalent color reproduction produces high quality images.
There is another type of color reproduction that can enhance images beyond equivalent reproduction. Preferred color reproduction is defined as a reproduction in which the colors depart from equality of appearance to those of the original, either absolutely or relative to white, in order to give a more pleasing result to the viewer. Some preferred color enhancements are based on the concept of memory colors. Research has shown, that our memory of certain colors, for example skin colors, foliage and blue sky, deviates from the actual color. Memory colors often have different hues and enhanced colorfulness compared with the actual colors. There is evidence that viewers prefer reproductions that are closer to the memory color than to the actual color. Several researchers have tried to obtain optimum positions for these colors in controlled psychophysical experiments. However, the results often contradict each other, and it has been shown that color preferences may change over time as systems with larger color gamuts become available. The concept of memory colors has never been systematically incorporated into the design of color reproduction systems.
While the principles of preferred color reproduction, including the importance of hue reproduction and memory colors, were recently summarized by Hunt in a general fashion (R. W. G. Hunt, xe2x80x9cHow To Make Pictures and Please Peoplexe2x80x9d, The Seventh Color Imaging Conference, ISandT, Springfield, Va., 1999), it is not obvious how to make images according to these principles. Our experience has shown that it is impossible to produce images that embody all the principles of preferred color reproduction using conventional silver halide film/paper systems.
Current optical and digital photofinishing systems produce hues of reproduced colors that change as a function of lightness and chroma, thus giving the reproductions a somewhat unnatural appearance. FIG. 1 shows an example of the hue reproduction capabilities of a current consumer color negative/positive system in terms of a CIELAB a*/b* plot. For demonstration purposes the CIE 1976 a,b chroma, C*ab, was maintained at the original color position. The tails of the arrows denote the original color while the heads of the arrows (symbols) show the reproduced color. In this diagram, colors of constant CIE 1976 a,b hue angle, hab, fall along lines that emanate from the origin (a*=0, b*=0). The abscissa approximately corresponds to the green-red axis, while the ordinate represents the blue-yellow axis. Colors of constant CIE 1976 a,b chroma are represented by concentric circles around the origin. FIG. 1 shows that hues of colors of similar original hue angles may change in opposite directions. Furthermore, hue angle errors of saturated (high chroma) colors are often so large, that a reproduced color may cross a color name boundary. FIG. 1 for example suggests that saturated greens might be reproduced yellow.
One of the important criteria for viewer satisfaction in photographic reproductions is the correspondence between the color stimuli in the original scene compared to those of the reproduction. We find that viewers generally prefer to have high quality images with pleasing tone reproduction, pleasing hues, and high colorfulness while maintaining good skin tone. Technological advances have been made over the years in photographic films by improving spectral sensitivities, incorporating more chemical enhancement in photographic papers by increasing the paper contrast, and in the whole system by co-optimizing film and paper spectral sensitivities and dyes. Some current methods for making color reproductions produce fairly bright colors and offer reasonable skin tone reproduction; however, there have been limitations on the extent to which color enhancement can be employed. Conventional silver halide photographic systems are subject to limitations imposed by optically printing one chemically developed material onto another chemically developable material. As a result, we find that they generally do not reproduce the scenes in a way that is most preferred by the viewer.
Aside from color enhancement, the quality of image reproductions is also affected by the tone scale function or tone mapping employed to reproduce the density variations that make up an image. It has previously been discovered that the use of a preferential tone scale function or mapping as described generally in U.S. Pat. No. 5,300,381, issued Apr. 5, 1994 to Buhr et al., entitled Color Image Reproduction of Scenes with Preferential Tone Mapping, can be utilized to provide a reproduced image that is perceived by the viewer to be a reproduction of the original scene preferred to that previously obtainable. Buhr et al. also provided a solution to the problem of producing pleasing skin tones in combination with high color saturation, as described in U.S. Pat. No. 5,528,339, issued Jun. 18, 1996, entitled Color Image Reproduction of Scenes with Color Enhancement and Preferential Tone Mapping.
Many natural scenes photographed under ambient lighting conditions result in photographic images which have a luminance dynamic range that far exceeds the dynamic range of conventional display systems. For example, photographic images taken in sunny outdoor conditions can have 10 or more photographic stops of recorded information while photographic paper can reproduce approximately seven photographic stops of information. In digital imaging systems, scene dependent tone scale function algorithms may be employed to reduce the dynamic range of the source digital image thus providing a better match of the processed digital image to the dynamic range capabilities of the output medium.
One such scene dependent tone scale algorithm is described by Alkofer in U.S. Pat. No. 4,731,671, issued Mar. 15, 1988, entitled Contrast Adjustment in Digital Image Processing Method Employing Histogram Normalization. Alkofer discloses a method that calculates a tone scale function based on the pixels of the source digital image. The method involves calculating the standard deviation of a sub-sample of pixels selectively sampled from spatially active regions within the source digital image, calculating a histogram of these sampled pixel values, and calculating a tone scale function which when applied to the source digital image will result in a processed digital image which has a statistically normalized histogram. The resulting processed digital images using Alkofer""s method have dynamic ranges that are mapped to the intended output medium. As a consequence of the scene dependent tone scaling processing, the color reproduction of the processed images can be affected. Furthermore, depending on how the tone scale function is applied to the source digital image, the color reproduction of the processed images will be different.
The prior improvement in scene dependent tone scaling, tone mapping and color enhancement, has provided a degree of preferred reproduction of color images, but the use of tone mapping alone has not enabled the full extent of improvement desired by the viewer, in particular as far as hue reproduction is concerned. Recently, digital printing (e.g. the Digital Minilab Frontier 350 available from the Fuji Photofilm Company USA) and digitally-modified optical-printing (e.g. Agfa MSP DIMAX(copyright) printer available from Agfa A.G.) photofinishing systems have been introduced. These systems have introduced improvements in tone reproduction but have done little to improve color reproduction. Moreover, it has not been fully appreciated that the preferred visual reproduction does not usually correspond to the most colorimetrically accurate rendition. There is a need, therefore, for an improved image processing method that produces improved color reproduction in conjunction with scene dependent tone scaling algorithms.
The need is met according to the present invention by providing a method for enhancing the hue and lightness characteristics of a digital color image, the digital color image having pixel values from which digital luminance and color difference values can be deduced, that includes the steps of: deducing digital luminance and digital color difference values for pixels of the digital color image; using the pixels of the digital color image to calculate an image dependent transform; using the image dependent transform to modify the digital luminance values for pixels of the digital color image to form modified luminance values; calculating a color transform that modifies the original digital color difference values in a manner that consistently and smoothly moves the values toward or away from predetermined digital color difference values; and using the color transform to modify the original color difference values for pixels of the digital color image to produce modified color difference values.
This invention combines preferred color reproduction with scene-dependent tone scaling. As a result, higher color quality can be obtained compared with using any of the two algorithms in isolation. Preferred color mappings encompass generally accurate hue reproduction apart from a few selected regions of color space, where hues are modified in designed fashion, in order to produce reproductions that are highly preferred by customers. In addition, the colorfulness and the tone scale of the reproduction can be modified to produce images according to viewer preference. In preferred color manipulations, tone scale functions are usually implemented as a global transformation that do not take into account the dynamic range of the scenes. As a result, not all scene tones in high-dynamic range scenes can be reproduced on limited dynamic range output devices and media as viewers see them. This shortcoming of preferred color manipulations can be addressed by scene-dependent tone scaling algorithms that map scene tones into a range that can be reproduced by the intended output medium or device. In order to retain the advantages of hue control that are part of the preferred color manipulations, the tone scale manipulations must be implemented in a way that leaves hue unaltered. This can be achieved by applying the tone scale function to the luminance channel of a luminance/color difference space.
The primary concept of the present invention involves the coordination of a sub-system for scene dependent tone scale adjustment and a sub-system for adjusting the color hue and color chroma adjustment on the basis of color space coordinates. Specifically, the present invention involves the steps of: a) transforming a color digital image into a luminance-color difference scene space representation; b) applying a scene dependent tone scale function (in the form of a LUT) to the luminance channel data creating a tone scale adjusted digital image; c) transforming the tone scale adjusted digital image into an approximately perceptually uniform representation (such as CIE Lab); d) adjusting the hues of the tone scale adjusted digital image based on the CIELAB L*,a*,b* pixel values; e) adjusting the chroma of the tone scale adjusted digital image based on the CIELAB L*,a*,b* pixel values; and f) optionally applying an additional scene-independent tone scale function that may modify lightness alone, or lightness and chroma.